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SUMMARY 

The  purpose  of  the  study  reported  here  was  to  develop  and  explore  new 
techniques  and  procedures  to  aid  in  the  design  and  assessment  of  reentry 
vehicles  for  survival  against  X-rays  from  exoatinospheric  nuclear  bursts. 

At  present,  a  primary  source  of  design  information  for  reentry  vehicle 
hardness  against  X-rays  is  underground  nuclear  testing.  With  the  advent 
of  a  comprehensive  nuclear  test  ban  treaty,  this  source  of  proof  data  will 
no  longer  be  available. 

To  prepare  for  the  day  when  underground  nuclear  test  data  would  no 
longer  be  available,  Defense  Nuclear  Agency  (DNA)  initiated  a  program  to 
develop  a  Nuclear  Hardness  Evaluation  Procedure  (NHEP) .  From  this  program 
has  evolved  the  NHEP  methodology,  developed  by  Effects  Technology,  Inc., 
and  a  consortium  of  cooperating  Agencies  and  Contractors.  This  methodology 
consists  of  a  step-by-step  general  procedure  for  assessing  reentry  vehicle 
hardness,  supported  by  a  nuclear  weapons  effects  data  base. 

An  important  tool  which  has  been  utilized  to  demonstrate  the  effective¬ 
ness  of  the  NHEP  methodology  is  the  TRW-developed  Failure  Analysis  by 
Statistical  Techniques  (FAST)  code.  The  FAST  code  calculates  the  system 
probability  of  survival  and  associated  confidence  level  for  a  system  sub¬ 
jected  to  a  defined  environment.  During  two  previous  studies.  TRW  developed 
the  use  of  FAST  as  a  final  step  in  the  NHEP  methodology. 

In  addition,  new  design  optimization  techniques  were  developed  to 
identify  optimal  vehicle  design  configurations  to  assure  maximum  system 
hardness  or  minimum  system  weight  at  a  prescribed  probability  of  survival 
and  confidence  level.  These  optimization  techniques  were  incorporated  into 
a  new  computer  code,  BASH  (Balanced  System  Hardness),  and  the  utility  of 
these  design  optimization  techniques  was  verified  by  subsequent  review  of 
the  BASH  optimized  design  configurations,  both  by  FAST  code  assessment  and 
by  engineering  analysis. 


1 


The  present  study  is  an  extension  of  those  previously  reported  NHEP 
efforts.  In  the  current  study,  the  FAST  methodology  was  extended  to  pro¬ 
vide  new  techniques  for  estimating  the  potential  benefits  of  testing 
under  uncertainty.  A  new  computer  code  named  ELF,  Evaluation  of  Likelihood 
Function,  was  developed  to  provide  a  thorough  evaluation  of  test  results. 

To  provide  a  demonstration  of  the  ELF  code,  this  code  was  used  to  assess 
the  graphite  resin  magnetic  flyer  plate  test  data. 

A  conditional  probability  version  of  the  FAST  code,  FAST7 ,  was 
developed  which  shows  great  promise  for  test  planning  and  evaluation.  In 
addition,  as  a  part  of  this  contractual  effort,  TRW  provided  consultation 
and  support  to  Lockheed  Missiles  and  Space  Company  (LMSC)  in  the  FAST 
assessment  of  a  Reference  reentry  vehicle  design. 
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PREFACE 


This  final  report  documents  the  work  completed  by 
TRW  Defense  and  Space  Systems  Group  under  the  sponsorship  of 
the  Defense  Nuclear  Agency  Contract  DNA001-78-C-0135,  entitled 
"Cost/Benef it  Analysis  of  Reentry  Vehicle  Hardness  Testing 
Using  the  FAST  Methodology".  The  overall  period  of  technical 
performance  of  this  project  was  from  23  January  1978  to 
30  March  1979.  The  TRW  Project  Manager  for  this  work  was 
Dr.  John  J.  Farrell,  replaced  by  Mr.  Leo  H.  Donahue  in  the 
closing  months  of  the  project.  The  study  was  conducted  within 
the  Ground  Systems  and  Technology  Department,  under  the  direction 
of  Dr.  Peter  K.  Dai.  The  authors  are  indebted  to  Dr.  Charles  Stein 
and  Lt  Erv  Copus,  AFWL/DYV,  for  technical  direction,  and  to 
Mr.  Donald  J.  Kohler,  Contracting  Officer  DNA/SPAS,  for  numerous 
helpful  suggestions.  In  addition,  the  authors  wish  to  thank 
Dr.  E.  A.  Fitzgerald  of  McDonnell  Douglas  Astronautics  Co.  for 
his  support  and  suggestions  throughout  this  study  effort,  and 
M  r.  Richard  F.  Walz  of  Lockheed  Missiles  and  Space  Company  for 
his  help  in  the  reentry  vehicle  assessment  effort. 

For  convenience,  this  final  report  has  been  organized  in 
two  volumes:  Volume  1,  Applications  (Classified  Secret);  and 
Volume  2,  Methods  (Unclassified).  The  authors  responsible  for 
the  individual  sections  of  the  report  are  as  follows:  Volume  1, 
Sections  1  and  2,  L.  H.  Donahue,  Section  3,  G.  M.  Teraoka; 

Volume  2,  Sections  1  to  3,  L.  H.  Donahue,  Section  4,  J.  J.  Farrell. 
In  addition,  significant  contributions  were  made  to  all  sections 
by  Dr.  Farrell. 
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1.0  INTRODUCTION 

1.1  Background 

The  purpose  of  the  study  reported  here  was  to  develop  and  explore  new 
techniques  and  procedures  to  aid  in  the  design  and  assessment  of  reentry 
vehicles  for  survival  against  X-rays  from  exoatmospheric  nuclear  bursts. 

At  present,  a  primary  source  of  design  information  for  reentry  vehicle 
hardness  against  X-rays  is  underground  nuclear  testing.  With  the  advent  of 
a  comprehensive  nuclear  test  ban  treaty,  this  source  of  proof  data  will  no 
’  nger  be  available. 

To  prepare  for  the  day  when  underground  nuclear  test  data  would  no 
longer  be  available.  Defense  Nuclear  Agency  (DNA)  initiated  a  program  to 
develop  a  Nuclear  Hardness  evaluation  Procedure  (NHEP) .  From  this  program 
has  evolved  the  NHEP  methodology,  developed  by  Effects  Technology,  Inc., 
and  a  consortium  of  cooperating  Agencies  and  Contractors.  This  methodology 
consists  of  a  step-by-step  general  procedure  for  assessing  reentry  vehicle 
hardness,  suppe  rted  by  a  nuclear  weapons  effects  data  base. 

An  impou-.ant  tool  which  has  been  utilized  to  demonstrate  the  effective 
ness  of  the  NHEP  methodology  is  the  TRW- developed  Failure  Analysis  by 
Statistical  Techniques  (FAST)  code.  The  final  step  in  the  NHEP  methodology 
is  the  integration  of  the  data  generated  and  compiled  using  this  procedure, 
and  the  evaluation  of  reentry  vehicle  hardness  at  a  prescribed  confidence 
level.  The  FAST  code  is  the  analytical  tool  selected  for  this  task. 

The  FAST  code  calculates  the  system  probability  of  survival  and 
associated  confidence  b'Vwl  :  o-  a  system  subjected  to  a  defined  environment 
Hardness  probability  cq.in!  ions  specified  for  the  components  of  a  system  are 
inf  ‘grated  by  a  series  c(  Monte  Carlo  calculations  to  provide  a  hardness 
pro  lability  statement  for  the  system.  The  FAST  code  provides  the  means  by 
which  these  data  on  reentry  vehicle  component  hardness  and  hardness  uncer¬ 
tainty  can  be  combined  to  estimate  the  vehicle  hardness  level  and  the 
associated  probability  of  survival  and  confidence  level. 

During  two  previous  studies,  TRW  developed  the  use  of  FAST  as  a  final 
step  in  the  NHEP  methodology.  A  major  contribution  to  those  efforts  was 
made  by  McDonnell  Douglas  Astronautics  Company  (MDAC) ,  by  applying  the 
Nuclear  Hardness  Evaluation  Procedures  to  determine  the  reentry  vehicle 
component  hardnesses,  and  the  uncertainties  in  these  hardness  estimates. 
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These  data  were  then  used  by  TRW  to  develop  probabilistic  models  of 
component  hardnesses  and  to  integrate  these  hardness  models  with  the  PAST 
code  to  obtain  vehicle  system  hardness  estimates  with  their  associated 
probability  of  survival  and  confidence  levels. 

In  addition,  new  design  optimization  techniques  were  developed  to 
identify  optimal  vehicle  design  configurations  to  assure  maximum  system 
hardness  or  minimum  system  weight  at  a  prescribed  probability  of  survival 
and  confidence  level.  These  optimization  techniques  were  incorporated  into 
a  new  computer  code,  BASH  (Balanced  System  Hardness) ,  and  the  utility  of 
these  design  optimization  techniques  was  verified  by  subsequent  review  of 
the  BASH  optimized  design  configurations,  both  by  FAST  code  assessment  and 
by  engineering  analysis. 

1.2  Scope  and  Objectives 

The  present  study  is  an  extension  of  those  previously  reported  NHEP 
efforts.  In  the  current  study,  the  FAST  methodology  was  extended  to  pro¬ 
vide  new  techniques  for  estimatincr  the  potential  benefits  of  testing 
under  uncertainty. 

A  new  computer  code  named  ELF,  Evaluation  of  Likelihood  Function  was 
develooed  during  the  study  to  provide  a  thorough  evaluation  of  test 
results.  To  orovide  a  demonstration  of  the  ELF  code,  this  code  was 
used  to  assess  the  graphite  resin  magnetic  flyer  plate  test  data.  In 
addition,  as  a  part  of  this  contractual  effort,  TRW  provided  consultation 
and  support  to  Lockheed  Missiles  and  Space  Company  (LMSC)  in  the  FAST 
assessment  ot  a  Reference  reentry  vehicle  design. 

For  convenience,  the  final  report  of  these  activities  has  been  subdi¬ 
vided  into  two  volumes.  Volume  I,  Applications,  describes  the  results  of 
the  evaluations  performed  on  the  graphite  resin  magnetic  flyer  plate  tests 
and  their  impact  on  predictions  of  reentry  system  performance.  Also  in¬ 
cluded  in  Volume  I  are  the  results  of  the  analyses  performed  on  the  LMSC 
Reference  reentry  vehicle.  Volume  I  is  classified  Secret. 

Volume  II  of  the  Final  Report,  Methods,  describes  in  detail  the  devel¬ 
opment  of  the  statistical  and  analytical  techniques  that  were  evaluated 
during  this  study,  and  the  methods  that  were  incorporated  into  the  ELF  code. 
Also  reported  in  this  volume  is  a  new  version  of  the  FAST  code,  FAST7 , 
Conditional  Probability  FAST,  which  was  developed  during  this  effort, 
and  which  shows  great  promise  for  test  planning  and  evaluation. 
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2.0  TEST  PLANNING  AND  EVALUATION 

Test  planning  and  evaluation  is  an  important  area  of  decision  making 
under  uncertainty.  The  reentry  vehicle  designer  is  often  faced  with  the 
weighty  question  of  whether  to  proceed  with  a  particular  test  or  test  series 
or  not.  To  answer  this  question,  he  must  be  able  to  determine  both  the  cost 
of  the  test,  and  the  cost  impact  of  the  probable  improvement  in  system  per¬ 
formance  that  can  be  realized  as  a  result  of  the  tests. 

Fortunately,  these  two  aspects  of  the  problem  are  separable,  and  can 
be  decoupled  without  seriously  degrading  the  outcome  of  the  analysis.  The 
determination  of  testing  costs  is  accomplished  by  cost  estimators  using 
standard  estimating  techniques.  Likewise,  the  cost  savings  that  result 
from  system  performance  increases  involve  the  judgment  of  trained  cost 
estimators.  The  present  study  does  not  address  the  issue  of  cost  estimating, 
but  rather  is  intended  to  provide  a  methodology  to  assess  in  a  meaningful 
way  the  improvement  of  system  performance  that  may  result  as  a  consequence 
of  testing. 

In  an  uncertain  world,  the  performance  of  a  system  may  be  specified  in 
terms  of  its  ability  to  survive  under  a  given  load,  expressed  as  probability 
of  survival  and  associated  confidence  level,  and  the  system  cost  or  weiqht 
required  to  achieve  this  survivability.  An  improvement  in  system  performance 
may  be  expressed  in  terms  of  an  improvement  in  any  of  these  criteria. 

The  complete  characterization  of  the  probabilistic  response  of  a  typi¬ 
cal  system  or  component  to  a  univari  e  stimulus  is  given  by  the  response 
surface  illustrated  in  Figure  1.  This  surface  shows  the  probability  of 
survival,  P  ,  of  thesystem  as  a  function  of  the  load,  S,  or  stimulus  level 
applied  to  the  system,  over  a  range  of  confidence  levels,  C. 

Confidence  is  the  perceived  probability  that  at  least  the  proportion  of 
the  population  indicated  will  survive  the  given  load  showTn  by  the  correspond¬ 
ing  probabi  1  i  t.y-ve  r  sus  -  load  plane.  The  best  estimate  curve  of  probability 
of  survival  versus  load  is  usually  taken  to  be  at  the  50— percent  confidence 
plane.  In  Figure  1,  probabi 1 ity-of-survival-versus-load  planes  are  indi¬ 
cated  for  confidence  values  of  0.10,  0.50,  and  0.90.  The  projection  of 
these  intersections  onto  a  sinqle  plane  produces  the  useful  plot  shown  on 
the  right  side  of  Figure  1. 
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The  response  surface  shown  in  Figure  1  is  predicated  on  the  premise 
that  there  are  two  distinct  types  of  uncertainties  affecting  the  response 
of  the  system:  uncertainties  which  are  random,  and  uncertainties  which  are 
systematic,  and  that  random  uncertainties  determine  the  probability  of  sur¬ 
vival  of  the  system  modeled,  while  systematic  uncertainties  affect  the 
confidence  ascribed  to  that  probability  of  survival.  Although  this 
representation  of  the  consequences  of  variation  analysis  is  admittedly 
simplistic,  experience  has  demonstrated  that,  in  practice,  it  provides  a 
useful  method  of  accounting  for  the  effects  of  uncertainties. 

Random  uncertainties  are  the  result  of  the  many  indeterminable 
differences  that  occur  from  one  system  to  the  next,  or  from  one  run  to  the 
next.  These  differences  exist  at  some  level  no  matter  how  carefully  the 
system  is  specified,  or  how  rigorously  quality  control  is  excerised  in 
the  manufacture  or  operation  of  the  system. 

Systematic  uncertainties  in  the  response  of  the  system  exist  because 
of  a  lack  of  complete  knowledge  of  the  underlying  processes  affecting  the 
variable  under  analysis.  If  this  ignorance  could  be  reduced  by  additional, 
more  refined  research  or  testing,  or  by  implementing  more  sophisticated 
analytical  procedures,  the  amount  of  systematic  uncertainty  affecting  the 
probability  of  survival  could  be  reduced. 

Random  and  systematic  uncertainties  have  distinctly  different  effects 
on  the  survivability  of  the  system.  Random  variations  in  system  properties 
cause  some  systems  to  be  harder,  and  others  to  be  softer.  These  variations 
tend  to  average  out  when  the  probability  of  survival  of  a  large  number  of 
systems  is  considered.  Thus,  while  random  variations  cause  a  dispersion  in 
the  hardness  distribution,  they  do  not  generally  affect  the  central  tendency 
of  the  distribution. 

On  the  other  hand,  systematic  uncertainties  represent  potential  dif¬ 
ferences  between  the  real  performance  of  a  system  and  its  predicted 
performance  based  on  analytical  models  and  model  parameter  values.  These 
potential  differences  dictate  the  probability  of  a  bias  or  shift  in  the 
hardness  distribution.  Thus,  because  systematic  uncertainties  obscure  the 
central  location  of  the  hardness  distribution,  these  uncertainties  reduce 
the  system  confidence  shown  in  Figure  1. 
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The  first  step  in  the  analysis  of  the  benefits  resulting  from  testing 
is  the  selection  of  possible  tests  to  be  considered.  The  possibilities 
include  both  system  tests  and  component  tests.  However,  it  is  nearly  always 
true  that  component  tests  will  be  less  expensive  than  system  tests.  Thus, 
the  designer  will  ordinarily  elect  to  analyze  tests  of  components,  search¬ 
ing  especially  for  those  components  having  large  systematic  uncertainties 
which  intuitively  would  tend  to  have  the  greatest  effect  on  system 
uncertainty . 

When  candidate  tests  have  been  selected  for  analysis,  sensitivity 
studies  are  performed  to  estimate  the  impact  on  system  performance  of 
each  of  the  tests  selected.  Using  the  FAST  code,  the  effect  of  the  com¬ 
ponent  uncertainties  on  system  performance  is  simulated  by  making  a  series 
of  runs  while  varyinq  the  uncertainties  of  the  selected  component. 

These  sensitivity  studies  will  provide  results  which  may  be  expressed, 
for  example,  as  shown  in  Figure  2.  For  a  constant  level  of  probability  of 
survival,  confidence,  and  load,  the  cost  or  weight  of  the  system  will  de¬ 
crease  as  systematic,  or  bias,  uncertainty  for  a  given  component  decreases. 
Thus,  by  decreasing  the  systematic  uncertainty  associated  with  the  com¬ 
ponent,  it  is  possible  for  the  system  to  display  a  lower  cost  at  the  same 
load  level,  or  the  same  cost  or  weight  at  a  higher  load  level. 


Figure  2.  Component  Uncertainty  Sensitivities 
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An  alternative  is  the  use  of  the  Balanced  System  Hardness  (BASH)  code 
for  performing  the  sensitivity  analyses  without  the  need  for  the  Monte  Carlo 

procedure  which  is  the  basis  of  the  FAST  code.  The  BASH  code,  besides  being 

more  quick  and  efficient,  provides  directly  an  optimization  of  the  mix  of 

(2) 

components  in  the  final  system  design. 

To  determine  the  utility  of  a  given  test,  it  is  then  necessary  to  esti¬ 
mate  the  change  in  component  systematic  uncertainty  that  would  result  from 

additional  test  data.  By  the  use  of  the  sensitivity  plots  of  Figure  2,  the 
system  improvement  can  be  evaluated  as  a  function  of  these  changes  in 
component  uncertainty. 

Two  different  approaches  for  the  analysis  of  the  effects  of  testing 
were  investigated  during  this  study.  The  first  is  a  method  for  analysis 
of  test  data  that  utilizes  maximum  likelihood  estimation  of  the  distribu¬ 
tional  parameters.  This  method  is  embodied  in  a  new  computer  code  named 
ELF,  Evaluation  of  Likelihood  Function.  The  second  approach  involves  the 
use  of  conditional  probability,  and  has  been  incorporated  into  a  new  version 
of  the  FAST  code  named  FAST7. 

Both  of  these  computer  codes  can  be  used  to  objectively  evaluate  system¬ 
atic  uncertainties  based  on  test  results.  The  ELF  code  can  be  used  to 
analyze  both  attributes  (pass/fail)  and  variables  test  data  so  long  as  the 
test  simulation  is  perfect.  Where  there  is  uncertainty  in  the  simulation 
of  the  operation  conditions  by  the  test,  this  additional  uncertainty  must 
be  accounted  for  outside  the  ELF  code,  as  discussed  below. 

The  FAST7  code  is  a  powerful  tool  for  analyzing  the  effects  of  pass/fail 
data,  and  it  does  not  require  perfect  test  simulation.  In  addition,  FAST7 
can  be  used  at  subsystem,  or  system  levels,  and  can  analyze  correlations 
between  component  responses. 
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3.0  PARAMETRIC  ANALYSIS  METHOD 


The  first  requirement  for  a  testing  cost/benefit  analysis 
is  to  establish  reliable,  objective  techniques  for  reduction  of  raw  test 
data  to  usable  statistical  models.  The  criteria  for  measuring  the  success 
of  the  data  reduction  techniques  include  the  following:  (i)  they  must 
produce  results  which  can  be  reproduced  by  workers  throughout  the  com¬ 
munity;  (ii)  they  must  be  stable  in  the  sense  that  they  will  yield 
reasonable  results  even  if  only  limited  data  are  available;  and  (iii) 
they  must  provide  results  in  a  form  that  are  compatible  with  cost/benefit 
modeling . 

One  useful  methodology  developed  during  this  study  which  meets  the 
criteria  stated  above  is  the  so-called  parametric  analysis  described  here. 

This  method  depends  upon  the  estimation  of  the  statistical  parameters 
which  determine  the  distribution  of  the  test  variables.  A  later  section  of 
this  report  will  discuss  a  method  which  is  not  dependent  on  the  estimation 
of  parameters  to  determine  distributional  information  regarding  the  test 
variables.  This  is  the  conditional  probability  analysis  method. 

In  general  terms,  the  parametric  analysis  method  is  based  upon  the 
construction  and  application  of  mathematical  models  of  the  results  of 
repetitive  experiments.  These  models  make  possible  the  study  of  the  pro¬ 
perties  of  the  experiment,  and  predictions  about  the  outcome  of  future 
trials  of  the  experiment. 

The  conclusions  drawn  from  this  use  of  mathematical  modeling  are 
statistical  inferences,  and  the  process  involved  is  decision-making 
based  upon  probability.  Most  of  the  decisions  or  inferences  made  by 
statisticians  fall  into  one  of  two  categories;  either  they  involve 
the  estimation  of  the  parameters  of  a  frequency  function,  or  they  involve 
testing  some  distributional  assumption  about  the  frequency  function  selected 
for  the  model.  In  the  parametric  analysis  method  developed  during  this  study, 
both  types  of  statistical  decisions  are  utilized  in  analyzing  test  data. 

The  method  that  is  usually  preferred  by  most  statisticians  to  construct 
a  point  estimate  of  a  parameter  of  a  frequency  function  of  a  random 
variable  from  several  independent  observations  is  the  maximum  likelihood 
method.  By  definition,  the  maximum  likelihood  estimator  of  a  parameter 
of  a  frequency  function  is  the  estimator  that  maximizes  the  likelihood 
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function  in  terms  of  that  parameter.  The  likelihood  function  is  simply 
the  joint  frequency  function  of  the  several  independent  observations, 
formed  by  multiplying  together  the  frequency  functions  of  each  observation. 

A  statistical  test  of  a  distributional  assumption  provides  an 
objective  technique  for  assessing  whether  an  assumed  distributional 
model  provides  an  adequate  discription  of  observed  data.  The  following 
basic  steps  are  usually  involved:  (1)  a  figure  of  merit,  or  modulus, 
known  as  a  test  statistic  is  calculated  from  the  observed  data;  (2)  the 
probability  of  obtaining  the  calculated  test  statistic,  assuming  the  selected 
distributional  model  is  correct,  is  determined;  and  (3)  if  the  probability 
of  obtaining  the  calculated  test  statistic  is  low,  it  is  concluded  that  the 
assumed  distributional  model  does  not  provide  an  adequate  representation  of 
the  data. 

The  definition  of  low  depends  on  the  user's  preferences  and  the 
consequences  of  rejecting  the  model.  A  probability  of  0.10  or  less  is 
usually  said  to  be  low.  If  the  probability  associated  with  the  test 
statistic  is  not  low,  then  the  data  provide  insufficient  evidence  that  the 
assumed  model  is  adequate. 

Note  that  although  this  procedure  permits  a  distributional  model  to 
be  rejected  as  inadequate,  it  does  not  prove  that  the  model  is  correct.  The 
outcome  of  a  statistical  test  is  highly  dependent  upon  the  amount  of  avail¬ 
able  data:  the  more  data  there  are,  the  better  are  the  chances  of  identi¬ 
fying  an  inadequate  model.  If  too  few  data  points  are  available,  even  a 
distributional  model  that  deviates  grossly  from  the  assumed  model  frequently 
cannot  be  established  as  inadequate. 

Finally,  after  point  estimates  of  the  distributional  parameters  have 
been  determined,  and  the  distributional  hypotheses  utilized  in  the  analysis 
have  been  tested,  confidence  bounds  on  the  estimated  distribution  can  be  com¬ 
puted  based  on  the  test  data  sample  size  and  behaviour.  These  confidence 
bounds  act  as  a  buffer  on  the  estimated  distribution  function  to  account  for 
uncertainties  in  the  estimation  of  the  parameters. 

The  parametric  analysis  methods  outlined  here  were  developed  and 
codified  in  the  Evaluation  of  Likelihood  Function  (ELF)  computer  code  during 
this  study.  The  development  of  the  methodology,  and  its  application  to  a 
typical  data  analysis  problem  are  described  in  the  following  sections  of 
this  report. 
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3. 1  MAXIMUM  LIKELIHOOD  ESTIMATION 


Typically,  experimental  data  may  be  categorized  as  one  of  two  types, 
either  quantitative  or  quantal.  Quantitative  data  are  those  for  which  both 
the  stimulus,  or  load,  applied  to  the  test  specimen,  and  the  response  of  the 
specimen,  can  be  measured  quantitatively;  quantal,  or  attributes,  data  are 
those  data  for  which  the  response  is  not  quantitative,  but  rather  is  of  the 
all-or-nothing,  pass-or-fail,  type. 

Quantitative  data  are  comparatively  easy  to  analyze  statistically  be¬ 
cause  the  stimulus  for  each  data  point  is  associated  with  a  specific  quanti¬ 
tative  response.  Quantal  data,  on  the  other  hand,  present  special  problems 
in  analysis  because  the  responses  are  not  proportionally  related  to  the 
stimuli.  In  addition,  quantal  data  are  less  powerful  in  the  sense  that  each 
response  contains  less  information  than  a  corresponding  quantitative  test 
point . 

In  quantal  analysis,  each  individual  test  specimen  has  a  particular 
stimulus  level,  called  a  tolerance,  at  which  it  will  fail.  If  a  particular 
specimen  is  tested  at  a  stress  level  below  its  tolerance  level,  it  will 
survive  and  if  it  is  tested  at  a  stress  level  greater  than  its  tolerance 
level,  it  will  fail.  The  test  data  represent  a  sample  from  a  total  popu¬ 
lation  of  material  specimens  which  have  a  continuously  distributed  tolerance 
level.  It  is  this  tolerance  level  distribution  that  determines  the  prob¬ 
ability  of  failure,  P,  of  a  particular  specimen  at  a  given  test  level. 

Tables  1  and  2  contain  a  summary  of  magnetic  flyer  impact  test  results 

for  Tape  Wound  Carbon  Phenolic  (TWCP) /silicone  rubber/0.120  graphite  resin 

specimens.  These  data  were  assembled  by  E.  A.  Fitzgerald  of  McDonnell 

Douglas  Astronautics  Company  from  tests  conducted  as  a  part  of  the  Graphite 

Resin  Screening  Program,  the  Dynamic/Degraded  Properties  Program,  and  the 
( 3 ) 

NHEP  Program.  '  The  test  levels  are  given  in  terms  of  the  front  face 
pressure  on  the  TWCP,  which  is  an  approximate  measure  of  the  tensile 
stresses  inside  the  graphite  resin  composite.  The  criteria  for  pass  and 
fail  used  by  Fitzgerald  in  rating  the  performance  of  the  specimens  are: 

Pass,  no  more  than  one  hairline  crack  or  delamination  visible  in  the 
substructure  by  optical  or  X-ray  examination;  and  Fail,  any  damage  more 
extensive  than  that  described  above,  including  multiple  hairline  cracks  or 
delaminations,  single  open  cracks,  or  detached  spalls. 


Table  1.  Graphite  Resin  Test  Data 


TEST 

PRESSURE 

S 

LOG(S) 

RESULT 

DESCRIPTION 

1 

2.4000 

.87547 

PASS 

FLAT,  6DA  EPOXY  16R8P-2) 

2 

3.7000 

1.30833 

PASS 

FLAT,  GIA  EPOXY  (GRSP-2) 

3 

5.2000 

1.44844 

FAIL 

FLAT,  6DA  EPOXT  (GRSP-2) 

4 

2.4000 

.87547 

PASS 

FLAT,  6 DA  EPOXY  18RSP-2) 

5 

3.7000 

1.30833 

PASS 

FLAT,  GDA  EPOXY  (8RSP-2) 

6 

5.2000 

1.44844 

PASS 

FLAT,  GIA  EPOXY  (GRSP-2) 

7 

2.4000 

.87547 

PASS 

FLAT,  IIDAC  EPOXY  (8RSP-2) 

8 

3.7000 

1.30833 

PASS 

FLAT,  HDAC  EPOXY  (GRSP-2) 

9 

5.2000 

1.44844 

PASS 

FLAT,  HDAC  EPOXY  (6R8P-2) 

10 

4.0000 

1.58024 

FAIL 

FLAT,  GDA  EPOXY  (GRSP-3) 

11 

4.0000 

1.58024 

FAIL 

FLAT,  6DA  EPOXY  (6RSP-3) 

12 

3.7000 

1.30833 

FAIL 

FLAT,  GDA  EPOXY  1BRSP-3) 

13 

3.7000 

1.30833 

PASS 

FLAT,  GDA  EPOXY  (GRSP-3) 

H 

4.0000 

1.58024 

PASS 

FLAT,  HDAC  EPOXT  (GRSP-3) 

15 

4.0000 

1.70174 

PASS 

FLAT,  HDAC  EPOXY  (GRSP-3) 

14 

4.0000 

1.58024 

PASS 

FLAT,  HDAC  EPOXY  (GRSP-3) 

17 

8.7000 

2.14332 

FAIL 

FLAT,  HDAC  EPOXY  (GRSP-3) 

18 

2.4000 

.87547 

PASS 

FLAT,  HDAC  POLYIHIDE  (GRSP-3) 

1? 

2.0000 

1.04471 

PASS 

FLAT,  HDAC  POLYIHIDE  (6RSP-3) 

20 

4.0000 

1.58024 

PASS 

FLAT,  HDAC  POLYIHIDE  (GRSP-3) 

21 

8.7000 

2.14332 

FAIL 

FLAT,  HDAC  POLYIHIDE  (GRSP-3) 

22 

4.0000 

1.70174 

FAIL 

FLAT,  HDAC  POLYIHIDE  (GRSP-3) 

23 

2.4000 

.87547 

PASS 

RING,  6DA  EPOXY  (GRSP-3) 

24 

4.0000 

1.58024 

FAIL 

RING,  GDA  EPOXY  (6RSP-3) 

23 

3.7000 

1.30833 

PASS 

RING,  6DA  EPOXY  (GRSP-3) 

24 

2.4000 

.87547 

FAIL 

RING,  HDAC  EPOXY  (GRSP-3) 

27 

4.0000 

1.58024 

FAIL 

RING,  HDAC  EPOXY  (GRSP-3) 

26 

3.7000 

1.30833 

FAIL 

RING,  HDAC  EPOXY  (GRSP-3) 

,  >  .  . 
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Table  1.  Graphite  Resin  Test  Data  (Cont’d) 


TEST 

PRESSURE 

S 

L0G<S) 

RESULT 

DESCRIPTION 

29 

3.7000 

1 .30833 

FAIL 

RIN6, 

HUAC 

POLYINIDE  (6RSP-3) 

30 

1.9000 

.64185 

PASS 

FLAT, 

MDAC 

EPOXY  (DDPP) 

31 

1 .9000 

.64185 

PASS 

FLAT, 

MDAC 

EPOXY  (DDPP) 

32 

1  .9000 

.64185 

PASS 

FLAT, 

NDAC 

EPOXY  (DDPP) 

33 

'  .’000 

.64185 

PASS 

FLAT, 

NBAC 

EPOXY  (DDPP) 

34 

1.9000 

.64185 

PASS 

FLAT, 

MDAC 

EPOXY  (DDPP) 

35 

1.9000 

.64185 

PASS 

FLAT, 

MDAC 

EPOXY  (DDPP) 

36 

2.8000 

1 .02962 

PASS 

FLAT, 

MDAC 

EPOXY  (DDPP) 

32 

2.8000 

1 .02962 

PASS 

FLAT, 

MDAC 

EPOXY  (DDPP) 

36 

3.6000 

1.28093 

PASS 

FLAT, 

MDAC 

EPOXY  (DDPP) 

i? 

2.9000 

1.06471 

PASS 

FLAT, 

MDAC 

EPOXY  (DDPP) 

an 

2.9000 

1 .06471 

PASS 

FLAT, 

MDAC 

EPOXY  (DDPP) 

4  1 

2.4000 

.87547 

PASS 

ARC, 

MDAC 

EPOXY  (NHEP) 

42 

2.9000 

1 . 064 1 1 

FAIL 

ARC, 

MDAC 

EPOXY  (NHEP) 

43 

3.7000 

1.30813 

PASS 

ARC, 

MDAC 

EPOXY  (NHEP) 

44 

3.7000 

1.30833 

PASS 

ARC, 

MDAC 

EPOXY  (NHEP) 

45 

4.9000 

i. 58924 

FAIL 

ARC, 

MDAC 

EPOXY  (NHEP) 

46 

2.4000 

.87547 

PASS 

ARC, 

MDAC 

POLYINIDE  (NHEP) 

47 

2.9000 

1 .06471 

FAIL 

ARC, 

MDAC 

POL Y I MI DE  (NHEP' 

48 

3.7000 

1.30833 

PASS 

ARC, 

MDAC 

POLYINIDE  (NHEP) 

49 

3.7000 

1.30833 

FAIL 

ARC, 

MDAC 

POLYINIDE  (NHEP! 

50 

4.9000 

< .58974 

PAIL 

ARC, 

MDAC 

POLYINIDE  (NHEP) 

51 

' .6000 

.47000 

PASS 

RING, 

MDAC 

EPOXY  (NHEP) 

52 

1.6000 

.  47000 

PASS 

RIN6 , 

MDAC 

EPOXY  (NHEP) 

53 

2.9000 

1 .C64’1 

FAIL 

RING, 

MDAC 

EPOXY  (NHEP) 

54 

2.9000 

1.0647! 

PASS 

RING, 

MDAC 

EPOXY  (NHEP) 

55 

3.7000 

1.30833 

FAIL 

RING, 

MDAC 

EPOXY  'NHEP) 

56 

3.7000 

1.30833 

FAIL 

RING, 

MDAC 

EPOXY  (NHEP) 

57 

2.9000 

1 .06471 

PASS 

RING, 

MDAC 

POL Y I M I PE  (NHEP) 

58 

2.9000 

1 .06471 

PASS 

RING, 

MDAC 

POLYINIDE  (NHEP) 
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As  indicated  in  Table  2,  the  data  consist  of  a  total  of  58  tests,  with 
33  flat,  15  ring,  and  10  arc  specimens  having  been  tested.  Two  contractors 
are  represented,  McDonnell  Douglas  Astronautics  Company  (MDAC)  with  a  total 
of  45  specimens,  and  General  Dynamics  (GDA)  with  13  specimens.  Forty-five 
of  the  specimens  were  fabricated  from  epoxy  resin  reinforced  with  graphite 
fibers,  and  13  were  made  from  graphite-reinforced  polyimide. 

The  information  recorded  for  each  test  point  consists  of  the  pressure 
level  (S,  kbar)  induced  in  the  material  specimen,  and  the  quantal  response 
of  the  specimen,  i.e.,  whether  it  passed  or  failed  the  applied  stress. 

Also  shown  in  Table  1  are  the  logarithms  of  the  test  pressure  levels,  and  a 
description  of  the  material  sample. 

Table  3  shows  the  sample  data  from  Table  1  after  it  has  been  classi¬ 
fied.  Because  only  eleven  test  levels  were  represented  in  the  raw  data,  it 
was  possible  to  classify  these  data  at  point  values  of  the  independent 
variable  (rather  than  within  arbitrary  cell  boundaries).  At  each  test  level. 
Table  3  shows  the  test  level,  s,  the  logarithm  of  the  test  level,  the  number 
of  specimens  tested,  N,  the  number  of  specimens  failed,  R,  and  the  fraction 


Table  3.  Classified  Graphite  Resin  Test  Data 


CLASSIFIED  INPUT  DATS] 


8R0UP 

s 

LOS (S  > 

N 

R 

PF 

1 

1.4000 

0.47000 

2 

0 

0.00000 

2 

1.0000 

0.44183 

4 

0 

0.00000 

3 

2.4000 

0.87547 

8 

1 

0.12500 

4 

2.8000 

1.02942 

2 

0 

0.00000 

3 

2.9000 

1.04471 

9 

3 

0.33333 

4 

3.4000 

1.28093 

1 

0 

0.00000 

7 

3.7000 

1. 30833 

14 

4 

0.42857 

8 

4.9000 

1.58924 

9 

4 

0.44447 

9 

3.2000 

1.44844 

3 

1 

0.33333 

10 

4.0000 

1.79174 

2 

1 

0.50000 

11 

8.7000 

2.14332 

2 

2 

1 .00000 
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of  the  specimens  that  failed,  PF.  Classification  of  the  quantal  data,  and 
calculation  of  the  fraction  failed,  makes  it  possible  to  plot  these  data  in 
terms  of  stimulus  (stress)  and  response  (failure  probability) ,  as  shown  in 
Figure  3. 

The  next  step  in  the  statistical  analysis  of  these  data  is  to  fit  a 
predictive  model  to  the  data  points  shown  in  Figure  3.  This  is  done  by: 

(1)  assuming  a  frequency  function  for  the  data;  (2)  estimating  the  para¬ 
meters  of  the  frequency  function  by  the  maximum  likelihood  method;  and  (3) 
testing  the  assumptions  of  the  frequency  function  selected. 

Maximum  likelihood  estimating  consists  of  determining  the  maximum  of 
the  likelihood  function,  where  the  likelihood  function  is  the  joint  prob¬ 
ability  density  function  for  all  the  independent  observations.  For  the 
classified  data  of  Table  2,  a  Bernoulli  process  is  assumed  to  model  the 
response  of  the  specimens  at  each  test  level,  because  each  specimen  is 
assumed  to  respond  independently,  with  two  possible  outcomes  for  each  sample 
Then  the  probable  function  at  any  test  level  may  be  predicted  by  the  bino¬ 
mial  frequency  function 

f  =  Probability  function  for  the  test  level 
n  =  Number  of  data  points  at  test  level 
r  =  Number  of  failures  at  test  level 
P  =  Probability  of  failure  at  test  level 
Q  =  Probability  of  survival  at  test  level 
=  1  -  P 

The  likelihood  function,  which  is  the  joint  probability  function  for 
all  the  levels  tested,  is  expressed  as  the  product  of  the  binomial  prob¬ 
ability  functions  for  each  of  the  k  test  levels,  or 


A  lognormal  distribution  was  assumed  for  the  population  from  which  the 
test  data  shown  in  Table  1  were  drawn.  This  assumption  is  later  tested,  as 
discussed  below.  Utilizing  the  notation  of  Abramowitz (4)  and  Aitchison (5) , 
the  following  definitions  are  provided 


where : 
* 
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where 


x  =  log  e  s 


s  =  Test  level 


x  =  Logarithm  of  test  level 


For  a  lognormally  distributed  variable,  s<  the  transform  variable,  x 
is  normally  distributed  with  mean,  u  ,  and  standard  deviation,  a.  It  is 
convenient  to  introduce  the  standardized  normal  deviate 


z  = 


which  is  normally  distributed  with  a  zero  mean  and  a  unit  standard  deviation. 

The  frequency  function  for  the  standardized  normal  deviate,  z,  is 
given  by  the  familiar  relationship 


I  1  z2 

Z(2)  =  ±_  e  2  (5) 

vf2ir 

and  the  probability  of  failure  is  given  as  i_he  integral  of  equation  5, 


P(z)  =  J  Z(t)  dt  (6) 

-  CD 

The  likelihood  function,  L,  given  by  equation  2  describes  all  the 
probable  outcomes  of  sample  testing  in  terms  of  the  distributional  para¬ 
meters  p  and  o  .  A  typical  likelihood  function  is  illustrated  in  Figure  4. 
The  maximum  likelihood  estimates  of  the  distributional  parameters  are  ob¬ 
tained  by  determining  the  maximum  value  of  this  function. 

The  classical  method  of  determining  the  maximum  likelihood  estimates 
of  the  mean  and  standard  deviation,  and  a  ,  consists  of  differentiating 
the  likelihood  function  with  respect  to  each  of  the  parameters,  setting 
the  differentials  equal  to  zero,  and  solving  the  resulting  equations  for 
the  distributional  parameters.  That  is,  the  equations 


=  o  ;  Q  =  0 

d»  no 


(7,  8) 


A  A 

are  determined  and  solved  for  n  and  o. 
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Because  equation  2  is  not  amenable  to  simple  analytical  methods  of 
solution,  it  was  found  expeditious  to  solve  these  relationships  by  numerical 
methods.  Two  separate  numerical  methods  were  investigated  during  the  study, 
the  Probit  Analysis  method  and  a  search  technique.  Computer  routines  were 
designed,  implemented,  and  validated  for  each  of  these  methods,  and  a  com¬ 
bination  of  both  methods  was  incorporated  into  the  ELF  code. 

The  name  Probit  Analysis  has  come  to  mean  the  method  developed  and  pop¬ 
ularized  by  Bliss and  Finney ^ ^  .  This  method  was  originally  confined  to 
problems  in  biological  assay,  i.e.,  "the  measurement  of  the  potency  of  any 
stimulus,  physical,  chemical,  or  biological,  physiological  or  psychological, 
by  means  of  the  reactions  which  it  produces  in  the  living  matter"  It 

is  now  recognized  that  the  Probit  Analysis  method  can  be  applied  to  any 

(58) 

data  for  which  appropriate  data  transformations  can  be  determined 

The  basis  of  the  Probit  Analysis  method  is  transformation  of  the  test 
data  in  such  a  way  that  it  can  bo  fitted  with  a  best  (in  the  least  squares 
sense)  linear  regression  curve.  More  specifically,  where  quantal  data  are 

under  analysis,  the  data  are  classified,  and  the  fraction  of  samples  failing 
in  each  class  group  are  treated  as  the  dependent  variable. 

Traditionally,  the  independent  variable,  or  the  logarithm  of  the 
variable  when  appropriate,  has  been  taken  to  be  normally  distributed.  Even 
though  this  assumption  has  long  been  associated  with  Probit  Analysis,  it  is 
not  clear  that  it  is  essential  to  the  method.  When  the  normal  distribution 
is  assumed,  however,  the  values  of  the  independent  variable  are  conveniently 
transformed  into  standard  normal  deviates  to  linearize  the  data  fit. 

The  method  of  solution  utilized  in  the  Probit  Analysis  method  is 
facilitated  by  the  introduction  of  two  new  parameters ^ 5 ' ,  which  are  defined 
as 


a  =  -  p/ct  ( 9 ) 

0=1 /a  (10) 

Using  these  new  parameters,  equation  4  is  transformed  to  the  linear 
relationship 


z  =  a  +  fi  •  x 


(11) 
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and  Equations  7  and  8  are  replaced  by  the  relationships 


fa  ■  £  i  (r/n  -  p|  * 0  <12> 

i  =  1 

fe  -  £  *  i  '"">  -  p|  ■  0  1131 

i=  1 

which,  may  be  solved  iteratively  for  the  parameters  oc  and  0  .  Because  of 

the  popularity  of  this  method,  packaged  computer  routines  for  Probit 

( 9 ) 

Analysis  are  generally  available  in  the  scientific  community 

A  two-dimensional  search  technique,  augmented  by  a  quadratic  predictor 
algorithm,  was  also  developed  which  rapidly  and  efficiently  finds  the 
maximum  likelihood  estimates  for  g  and  c  .  The  two  methods  of  solution  for 
the  maximum  likelihood  estimates  of  the  parameters  of  the  tolerance  distri¬ 
butions  resulting  from  the  test  data  were  evaluated  for  accuracy  and 
efficiency.  It  was  found  that  the  two  techniques  complement  each  other  in 
the  sense  that  where  a  particular  data  set  gives  one  method  convergence 
problems,  the  other  method  usually  can  converge  quickly.  For  this  reason, 
the  ELF  ci.le  contains  a  hybrid  algorithm  which  utilizes  noth  techniaues. 

i  .  2  LTPOTtiELi LET  lb d 

Tests  of  the  two  distributional  assumptions  upon  which  the  parametric 
analysis  method  is  based  arc  incorporated  into  the  ELF  code.  The  first 
distributional  assumption  is  that  at  a  given  test  level,  each  test  may  be 
considered  to  be  the  result  of  a  Bernoulli  process.  Tnaf  is  to  say,  each 
test  specimen  responds  independently,  and  with  one  of  two  possible  outcomes, 
pass  or  fail,  with  probabilities  P  and  Q.  On  this  basis  the  test  data  at 
each  test  level  may  be  described  as  being  purely  binomially  distributed. 

This  hypothesis  is  tested  by  the  use  of  a  chi-square  test. 

The  second  distributional  assumption  that  was  made  was  that  the  test 
samples  were  drawn  from  a  population  that  is  lognormally  distributed. 
ALthough  the  assumption  of  this  frequency  function  is  not  essential  to  the 
parametric  analysis  method,  it  is  a  convenient  assumption  and  it  will  be 
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shown  to  have  much  utility  for  the  data  analyzed.  The  present  version  of 
the  ELF  code  assumes  the  lognormal  distribution,  and  tests  this  assump¬ 
tion  by  means  of  the  Wilk-Shapiro  W-Test. 


3.2.1  Chi-Square  Test 


The  most  commonly  used,  and  most  versatile  procedure  for  evaluating 
distributional  assumptions  is  the  chi-square  goodness-of-f it  test.  To  use 
this  test,  the  data  are  grouped  and  compared  to  the  expected  number  of 
observations  based  on  the  assumed  distribution.  The  major  advantage  of 
the  chi-square  test  is  its  versatility.  It  can  be  applied  in  a  simple  way 
to  test  any  distributional  assumption,  without  requiring  knowledge  of  the 
values  of  the  distribution  parameters.  Its  major  disadvantage  is  its  lack 
of  sensitivity  in  detecting  inadequate  models  when  few  observations  are 
available . 

2 

The  chi-square  statistic,  X  ,  is  defined  by  the  relationship 


X 


2 


(14) 


where  k  =  number  of  pairs  of  frequencies  to  be  compared 
°i  =  i  £h  observed  frequency 
ei  =  *  th  expected  frequency 

2 

A  value  of  X  of  zero  would  correspond  to  exact  agreement  between  observation 

2 

and  expectation,  while  increasingly  large  values  of  X  represent  increasingly 
poor  experimental  agreement. 

To  provide  a  continuous  measure  of  the  goodness  of  fit  implied  by  the 
2  ...  . 

X  statistic,  it  is  necessary  to  introduce  a  new  frequency  function  which 

2 

approximates  the  frequency  function  of  X  very  well  for  large  n.  This  new 

frequency  function  is  obtained  from  the  limit  of  the  moment  generating  func- 
2 

tion  of  X  when  n  approaches  infinity  as 


f  (X2  |  p)  -  2? 


(15) 


The  parameter  v  is  called  the  degrees  of  freedom  because  it  repre¬ 
sents  the  number  of  independent  pairs  being  compared.  Since  the  continuous 
chi-square  distribution  of  equation  15  is  only  an  approximation  to  the 
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discrete  chi-square  frequency  function,  judgment  must  be  exercised  with 
this  test  when  very  small  samples  are  examined.  Experience  and  theory 
indicate  that  the  approximation  is  usually  satisfactory,  provided  e.  >  5, 

and  k  5.  Below  these  levels  the  chi-square  test  results  are  suspect^1®^* 

The  chi-square  test  possesses  the  useful  property  that  it  is 
applicable  even  when  the  cell  probabilities  depend  upon  unknown  parameters, 
provided  that  the  unknown  parameters  are  replaced  by  their  maximum  likeli¬ 
hood  estimates  and  one  degree  of  freedom  is  deducted  for  each  estimated 

^  (10) 

Parameter 

For  the  binomial  relationship  of  equation  14,  the  cni-square  statistic 
may  be  more  easily  calculated  from  the  relationship 


v2  _  V*  n (r/n  -  P) 

X  -  Z-r  p  q 

i=l 

which  is  chi-square  distributed  with  k  -  2  degrees  of  freedom 


(16) 


(5) 


The  probability  level  associated  with  the  distributional  assumption 
is  given  by  the  integral  of  equation  15  as 


Q(X2  |v)  =  f  f  (t  I  v)  dt  (17) 

*^X2 

2 

This  integral  gives  the  probability  of  obtaining  the  calculated  value  of  X  , 
given  v  degrees  of  freedom.  Figure  5  shows  a  range  of  probability  values 
given  by  equation  17  as  a  function  of  X  and  v.  Equation  17  has  been  incor¬ 
porated  into  the  ELF  code,  providing  an  automatic  evaluation  of  the  chi- 
scuare  test  of  a  set  of  test  data. 


3 . 2 . 2  The  Wilk-Shapiro  W-Test 

The  w-test  is  an  effective  new  statistical  procedure  for  evaluating  the 
assumption  of  normality  (or  lognormality )  against  a  wide  range  of  alterna¬ 
tive  distributions,  even  if  only  a  relatively  small  number  of  observations 
are  available.  The  test  statistic  is  obtained  by  dividing  the^square  of  a 
combination  of  sample  order  statistics  by  the  sample  variance  .  The 
W-statistic  is  given  by  the  relationships 
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Chi-Sguare  Probabilities 


2  2 

W  =  t>  /s 


(18) 


1 

b  =  /■  a  ,  (x  .  ,  -x. ) 

n-i  +  1  n-i  +  1  l 

i=l 

■  -  ±  <«i  - 1)2 

where:  x^  =  ordered  sample  data  such  that  <_  x ^ 

i  =  n/2  for  n  even 
=  (n-l)/2  for  n  odd 

x  =  mean  of  data  sample 

and  the  values  of  the  coefficients,  a,  are  given  in  Table 


(19) 

(20) 


<  x,  <  ...  <  x 
—  j  —  —  n 


4. 


Figure  6  shows  the  relationship  between  sample  size,  W-statistic,  and 
the  probability  of  the  calculated  W-statistic.  Note  from  Figure  6  that  to 
obtain  0.50  probability,  a  W-statistic  of  greater  than  0.90  is  needed,  even 
for  the  smallest  sample  size. 


Hahn  and  Shapiro  have  approximated  the  results  in  Figure  6  with  an 

( 12 ) 

empirical  Johnson  S0  distribution  .  This  approximation  makes  possible 

the  calculation  of  the  standard  normal  deviate,  corresponding  to  the  prob¬ 
ability,  P,  of  a  given  W-statistic  by  the  equation 


z 


+  *  lQ9e 


(21) 


where  the  constants  V ,  V ,  and  e  are  given  in  Table  5.  This  relationship 
allows  the  continuous  calculation  of  the  approximate  probabilities  associ¬ 
ated  with  a  given  value  of  the  W-statistic,  and  is  incorporated  into  the 
ELF  code. 

3  *  3  CONFIDENCE  BOUNDS 

In  the  parametric  analysis  method,  confidence  bounds  on  the  estimated 
lognormal  distribution  are  calculated  from  the  test  data  by  classical 
means.  These  confidence  bounds  account  for  the  uncertainties  in  the  esti¬ 
mates  of  the  distributional  parameters  n  and  <r  that  are  the  result  of  the 
sample  size  and  behavior.  These  bounds  assume  that  the  test  is  perfectly 
calibrated,  and  that  it  simulates  the  operational  conditions  perfectly. 
Uncertainties  in  test  calibration  and  simulation  fidelity  represent  separate 
issues  discussed  later  in  this  volume. 
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Table  5.  Empirical  Constants  for  W-Test  Probabilities 


For  the  case  where  the  sampled  population  is  normally  distributed, 
confidence  bounds  on  the  estimation  of  m  and  a  may  be  calculated  by  the  use 
of  the  non-central  t-distribution  .  This  distribution  is  based  upon  the 
non-central  t  parameter,  t ' ,  which  is  defined  by  the  equation 


t'  =  -~~r  (2 

V*2/i' 

where:  z  =  standardized  normal  deviate  (see  equation  4) 

A  =  non-centrality  constant  parameter 
2 

X  =  chi-square  parameter  (see  equation  14) 
v  =  degrees  of  freedom  of  chi-square  parameter 


The  probability  density  function  for  non-central  t  is  given  by  the  expres¬ 
sion 


Ft  v  +  1) 


(«'+t,*>) 


Hh„ 


\  ~t'A 
Ulv+t'7)'. 


where 


Hh„ (x) 


u* 

ITT  exp 


I  “  *5(u+x)2  }  du 


(23) 

(24) 


is  the  Hh  function  as  defined  by  Fisher 

Given  that  the  test  data  sample  is  drawn  from  a  normal  population  with 
a  mean,  \i ,  and  a  standard  deviation,  cr,  let  z  (P)  represent  a  standard  normal 
deviate  such  that  a  proportion  P  of  the  population  will  lie  below  the  limit 
U(p)  =  M  +  z(P)  ct.  If  a  random  sample  of  n  observations  from  the  population 
is  available,  with  a  sample  mean  x  and  a  standard  deviation  S,  then  a  con¬ 
stant  Kc  can  be  defined  such  that  100C  percent  of  the  time  n  +  z  (P)a<  x  +  KcS. 
In  other  words,  there  is  confidence,  C,  that  at  least  the  proportion  P  of 
the  population  will  lie  below  x  +  KcS. 

This  probability/conf idence  statement  may  be  expressed 


C  =  Pr  |m  z(p)(j  <_  x  +  KcS l  (25) 

The  quantity  in  braces  may  be  rewritten  as 
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Jn  +  z(P)  Jn 
§7ct 


<  Kc  sTn 


(26) 


so  that  the  term  to  the  left  of  the  inequality  is  equivalent  to  non-central 
t  defined  in  equation  22.  Then  equation  25  may  be  expressed  as 


C  =  Pr  it'  |  v,&  <  Kc  J~n\ 

where  A  =  z (p) 

v  =  n  -  1 

For  convenience,  0  is  used  to  define  the  operation 


(27) 

(28) 
(29) 


e  (t'0  |  v,&)  =  /  f  |„,a)  dt' 


=  p..  If  <  tr 


(30) 

(31) 


Then  the  inverse  operation,  0  1,  gives  the  value  of  tg  corresponding  to  a 
specified  probability  level.  The  solution  of  equation  25  for  Kc  may  now  be 
written  as 


Kc  =  -pre"1!!  -  C|i«,A)  (32) 

\n 

This  equation  provides  the  standoff  distance  corresponding  to  a  specified 
probability  of  failure  (implied  by  A )  and  confidence  level,  C.  Using  the 
maximum  likelihood  estimates,  p  and  c,  as  the  sample  parameters,  the  actual 
limit  at  any  probability/confidence  limit  is  given  by 


a  a  _i 

U(P,C)  =  ^+_2_e  (1  -  Ck,A)  (33) 

yjn 

It  is  interesting  to  note  that  the  non-central  t-distribution  has  not 
received  wide  use  in  the  scientific  community.  This  is  partly  due  to  the 
fact  that  this  distribution  is  a  fairly  recent  development.  The  definitive 
exposition  of  the  theory  was  not  presented  until  the  work  of  Johnson  and 
Welch  was  published  in  1940  (13).  The  main  reason  this  distribution  has  not 
been  significantly  utilized,  however,  is  the  difficulty  encountered  in 
evaluating  the  functions  which  define  the  non-central  t-distribution. 
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The  non-central  t-distribution  is  often  confused  with  the  Student's 
t-distribution,  which  is  a  relatively  simple  function  to  evaluate.  The 
Student's  t-distribution  is  useful  in  evaluating  the  confidence  limits  of 
a  distribution  where  the  measure  of  dispersion,  a,  is  known,  and  only  the 
measure  of  central  tendency,  ji,  is  subject  to  uncertainty.  The  Student's 
t-distribution  is  a  special  case  of  the  non-central  t-distribution,  for  the 
situation  where.  A,  the  non-centrality  constant,  is  equal  to  zero. 

Modernly,  the  solution  for  the  integral  of  the  non-central  t-distri¬ 
bution  is  available  in  many  of  the  complete  software  libraries.  However, 
no  solutions  for  the  inverse  of  the  non-central  t  indicated  in  equation  31 
were  found  during  this  study,  either  in  software  libraries  or  in  the 
scientific  literature.  Solutions  for  both  the  probability  integral  and  the 
inverse  function  were  developed  and  incorporated  in  the  ELF  code  during 
thin  study. 

Numerical  solutions  for  the  probability  integral  of  the  non-central 
t-distribution  and  its  inverse  were  developed  by  the  following  methods. 
First,  the  probability  integral  was  evaluated  by  a  method  suggested  by 

(14) 

Owen  .  The  integral  of  equation  30  was  integrated  by  parts  repeatedly, 
yielding  distinct  expressions  for  even  and  odd  degrees  of  freedom,  v  .  For 
even  degrees  of  freedom,  the  terms  resulting  from  the  repeated  integrations 
are  evaluated  by  series  expansion  in  terms  of  the  standard  normal  integral. 
For  odd  degrees  of  freedom,  the  solution  is  expressed  in  terms  of  the 
standard  normal  integral  function  and  the  T-function  defined  by  Owen^15^. 

An  economical  numerical  solution  for  the  Owen  T-function  was  developed 
based  on  a  Gaussian  quadrative  method  given  by  Young  and  Minder  . 

An  efficient  numerical  method  was  developed  for  the  solution  of  the 

inverse  of  the  integral  of  the  non-central  t-distribution  involving  secant- 

guided  trial-and-error  iterations.  An  initial  estimate  of  the  solution  is 

provided  by  assuming  that  the  sample  is  normally  distributed,  using  a 

(13 ) 

formula  suggested  by  Johnson  and  Welch  .  Subsequent  estimates  are 
calculated  by  the  secant  method,  constrained  by  the  false  position,  or 
regula  falsi,  technique.  This  trial-and-error  method  is  greatly  accelerated 
by  transforming  the  values  of  the  iterations  into  the  normal  probability 
plane.  Even  though  this  transformation  is  only  approximate  for  the  non¬ 
central  t  probability  curves,  it  linearizes  these  S-shaped  curves 
sufficiently  so  that  the  secant  estimations  converge  quadratically . 
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Validation  of  the  software  resulting  from  these  studies  was  accomplished 

(17) 

by  reproducing  the  tables  of  Resnikoff  and  Lieberman  .  This  volume  is 
the  standard  resource  for  non-central  t-distribution  tables.  Unfortunately, 
the  accuracy  of  the  integral  tables  presented  is  only  four  significant 
figures,  and  the  tables  of  percentage  points  of  t  given  were  developed 
by  inverse  interpolation  in  the  probability  tables. 

The  ELF  code  rountines  were  also  tested  against  tables  by  Owens  ' , 

(19)  (8)  (20) 

Bombara  ,  Natrella '  ,  and  Pearson'  .  Finally,  the  code  was  self- 

tested  by  calculating  probabilities  and  then  inverse  probabilities,  to  see 
how  close  the  routines  could  come  to  the  starting  point. 

3 . 4  ELF  CODE  RESULTS 

Figure  7  shows  the  ELF  code  output  resulting  from  the  analysis  of  the 
total  quantity  of  data  available,  as  given  in  Table  1.  At  the  top  of  the 
page  are  listed  the  11  test  levels,  or  groups,  into  which  the  data  were 
classified.  Given  in  the  listing  are  the  group  number,  the  test  level,  S, 
the  logarithm  of  the  test  level,  the  total  number  of  tests  at  this  level, 

N,  the  number  of  failures  at  this  level,  R,  and  the  fraction  failed  at  this 
level,  PF  =  R/N. 

The  maximum  likelihood  estimates  of  the  parameters  of  the  population 
represented  by  this  sample  are  given  at  the  center  of  the  page.  The  median 
of  the  population,  m,  is  estimated  at  4.334  and  the  K-factor  characterizing 
the  random  variations  exhibited  by  the  samples,  K  ,  is  seen  to  be  2.719, 
the  mean  is  given  as  1.466,  and  the  standard  deviation  is  0.510. 

The  median  and  the  K-factor  represent  the  measures  of  central  tendency 
and  spread  of  the  data,  respectively,  in  the  natural,  or  untransformed, 
domain,  and  the  mean  and  the  standard  deviation  represent  the  same  proper¬ 
ties  of  the  data  sets  in  the  logarithm  domain.  These  parameters  are 
defined  mathematically  by  the  following  relationships 


x  =  log  s 

(34) 

where 

s  =  measure  of  load 

(here,  oressure  level,  kbars) 

x  =  logarithm  of  load 
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PROGRAM  ELF 
DATE:  30  MAR  79 
TIME!  00.30.00. 

ITEM:  ALL  AVAILABLE  DATA 


CLASSIFIED  INPUT  DATA: 


GROUP 

S 

L06(S> 

N 

R 

PF 

1 

1.6000 

0.47000 

2 

0 

0.00000 

2 

1  .9000 

0.64185 

6 

0 

0.00000 

3 

2.4000 

0.87547 

8 

1 

0.12500 

4 

2.8000 

1.02962 

2 

0 

0.00000 

5 

2.9000 

1 .06471 

9 

3 

0.33333 

6 

3.6000 

1.28093 

1 

0 

0.00000 

7 

3.7000 

1.30833 

14 

6 

0.42857 

8 

4.9000 

1 .58924 

9 

6 

0.66667 

9 

5.2000 

1.64866 

3 

1 

0.33333 

10 

6.0000 

1.79176 

2 

1 

0.50000 

1 1 

8.7000 

2.16332 

2 

2 

1.00000 
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MAXIMUM  LIKELIHOOD  PARAMETERS 

MEDIAN  =  A. 33355  MEAN  =  1.46639 

K-FACTOR  =  2.71935  STD  DEV  =  0.51041 


GOODNESS-OF-FIT  TESTS 

CHI-SGUARE  TEST  OF  VILK-SHAPIRO  TEST  OF 

BINOMIALITT  OF  DATA  LOGNORMALITY  OF  DATA 


CHI-SQUARE  =  4.50737  U-STATISTIC  =  0.98380 

DEG  OF  FREEDOM  =  9  SAMPLE  SIZE  =  11 

PROBABILITY  =  0.87497  PROBABILITY  *  0.98133 


Figure  7.  ELF  Code  Analysis  Summary 
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For  s  lognormally  distributed,  x  is  normally  distributed  with  a 
probability  density  function 


f  (x) 


(35) 


The  mean  of  the  log  of  load,  (l ,  is  defined  as 


f (x)  dx 


(36) 


and  the  standard  deviation  of  the  load,  <r ,  is  the  square  root  of  the  variance 

2  r°°  2 

°  -J  (x  “  V)  f(x)  dx 

■*  00 

The  median,  m,  is  given  implicitly  by  the  expression 
m 

0.5  =y*  f  (x)  dx  (38) 

—  oo 

Finally,  the  K-factor,  defined  by  the  expression 


K  =  exp  (1.96<r>  (39) 

is  a  factor  applied  to  the  median  to  give  the  upper  and  lower  bounds 
corresponding  to  95  percent  of  the  population. 

The  results  of  the  tests  of  the  distributional  assumptions  are  shown 
at  the  bottom  of  the  the  page.  The  chi-square  statistic  used  to  test  the 
assumption  of  binomiality  of  the  performance  of  the  specimens  at  each  test 
level  is  seen  to  be  equal  to  a  value  of  4.507.  The  probability  of  achiev¬ 
ing  a  value  of  chi-square  equal- to-or-greater  than  this  value,  with  9 
degrees  of  freedom,  is  87.5  percent.  In  other  words,  the  odds  against 
the  binomiality  assumption  being  correct  for  this  data  set  are  only  12.5 
percent  based  on  this  test.  The  W-statistic  computed  for  the  data  set  is 
0.984,  and  the  probability  level  associated  with  this  statistic,  for  a 
sample  size  of  11,  is  98.13  percent.  Thus,  the  tests  of  the  distributional 
assumptions  give  little  indication  of  nonconformity  of  this  data  set. 
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Figure  3  gives  tabulated  values  of  the  estimated  distribution  of  the 
probability  of  .ailure  for  the  graphite  resin  test  specimens.  These  values 
were  computed  from  the  sample  by  the  ELF  code  for  three  confidence  levels: 
10,  50,  and  90  percent.  The  50-percent  confidence  values  represent  the 
best  estimate  lognormal  distribution  for  this  population.  The  10  and 
90-percent  confidence  values  represent  the  systematic  uncertainty  based 
on  the  sample  size  and  variation,  assuming  perfect  test  calibration  and 
test  simulation.  These  confidence  bounds  are  theoretically  exact,  being 
computed  by  the  use  of  the  non-central  t-distribution,  as  described  above. 

Figure  9  shows  a  fragility  curve,  i.e.,  a  plot  of  probability  of 
failure  versus  test  level,  for  the  test  data  of  Table  1.  This  plot  illus¬ 
trates  the  grouped  data  points,  listed  in  Table  3,  together  with  the  10, 

50,  and  90-percent  confidence  curves  tabulated  in  Figure  8.  Also  shown  on 
this  plot  are  the  number  of  tests  in  each  group,  N,  and  the  number  of 
specimens  which  failed  in  each  qroup,  R. 

Tor  example,  at  the  test  level  of  6  kbar,  two  specimens  were  tested, 
with  one  passing  and  one  failing,  giving  an  experimental  failure  probability 
of  50  percent.  The  best  estimate  curve,  on  the  other  hand,  predicts  a 
failure  probability  for  the  total  population  at  this  test  level  of  73.7 
percent,  or  about  three  out  of  four.  At  the  test  level  of  3.7  kbar,  it  is 
seen  that  14  specimens  were  tested,  of  which  6  failed,  giving  an  experi¬ 
mental  failure  rate  of  42.9  percent.  In  comparison,  the  best  estimate 
prediction  of  failure  rate  at  this  test  level  is  seen  to  be  37.8  percent. 

As  intuition  would  predict,  the  larger  the  sample  at  any  given  test 
level,  the  more  nearly  the  measured  probability  of  failure  matches  the 
predicted  probability  of  failure.  With  the  maximum  likelihood  estimation 
method  employed  by  the  ELF  code  to  analyze  these  test  data,  the  best 
estimate  distribution  is  optimized  in  the  sense  of  minimizing  the  distance 
from  each  data  point,  on  the  basis  of  the  number  of  tests  at  that  point. 

In  other  words,  the  points  seen  plotted  in  Figure  3,  each  of  which  repre¬ 
sents  a  group  of  test  data,  were  weighted  in  the  analysis  according  to  the 
number  of  tests  in  the  qroup. 
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MEDIAN  = 

4. 3335 

MEAN 

*  1.4644 

K-FACTQR  = 

2.7194 

STD  DEV 

*  0.5104 

SIZE  * 

58 

ALL  AVAILABLE  DATA 

6R0UPS 

*  11 

TEST  LEVEL,  S 

(UNCERTAINTY  FACTOR,  KB) 

PROB 

CONFIDENCE 

<PF) 

10.0 

50.0 

90.0 

0.10 

0.7020 

(  1.2747) 

0.8949 

(  1.1997) 

1 .0737 

1.00 

1.0899 

(  1.2125) 

1.3215 

(  1.1584) 

1.5308 

2.00 

1 .2743 

(  1.1919) 

1.5188 

(  1.1448) 

1.7388 

3.00 

1.4067 

(  1.1794) 

1.6590 

(  1.1366) 

1.8857 

5.00 

1.6090 

<  1.1630) 

1.8714 

(  1.1261) 

2.1073 

10.00 

1.9763 

(  1.1399) 

2.2528 

(  1.1116) 

2.5043 

15.00 

2.2475 

(  1.1260) 

2.5533 

(  1.1034) 

2.8172 

20.00 

2.5269 

1  1.1162) 

2.8205 

(  1.0979) 

3.0966 

25.00 

2.7705 

(  (.1088) 

3.0718 

(  1.0942) 

3.3412 

30.00 

3.0048 

1  1.1030) 

3.3166 

(  1.0917) 

3.620* 

35.00 

3.2413 

(  1.0985) 

3.5607 

(  1.0903) 

3.8820 

40.00 

3.4780 

<  1.0951) 

3.8087 

<  1.0896) 

4.1502 

45.00 

3.7207 

<  1.0925) 

4.0649 

(  1.0898) 

4.4301 

50.00 

3.9729 

(  1.090B) 

4.3335 

(  1.0908) 

4.7270 

55.00 

4.2391 

(  1.0898) 

4.4199 

(  1.0925) 

5.0474 

60.00 

4.5251 

(  1.0896) 

4.9307 

(  1.0951) 

5.3995 

65.00 

4.8376 

(  1.0903) 

5.2742 

(  1.0985) 

5.7938 

70.00 

5.1866 

<  1.0917) 

5.6624 

(  1.1030) 

6.2457 

75.00 

5.5871 

{  1.0942) 

6.1135 

(  1.1088) 

6.7785 

80.00 

6.0645 

(  1.0979) 

6.6583 

(  1.1162) 

7.4320 

85.00 

6.6661 

(  1.1034) 

7.3550 

(  1.1260) 

8.2819 

90.00 

7.4991 

(  1.1116) 

8.3360 

(  1.1399) 

9.5024 

95.00 

8.9116 

(  1.1261) 

10.0353 

(  1.1630) 

11.6715 

97.00 

9.9591 

(  1.1366) 

11.3200 

(  1.1794) 

13.3505 

98.00 

10.8007 

(  1.1448) 

12.3648 

<  1.1919) 

14.7376 

99.00 

12.2677 

(  1.1584) 

14.2106 

(  1.2125) 

17.2301 

99.90 

17.4914 

<  1.1997) 

20.9850 

(  1.2747) 

26.7500 

Figure  8 .  ELF  Code  Estimated  Distribution 
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3.5  APPLICATION  OF  ELF  RESULTS 

The  fragility  of  the  graphite  resin  specimens  given  by  the  curves  of 
Figure  9  represents  a  nearly  complete  description  of  the  performance  of 
this  material  based  on  the  available  test  data.  Random  uncertainties  in 
the  material  performance  are  specified  by  the  random  K-factor,  KR, 
listed  at  the  top  of  the  figure. 

The  confidence  bounds  shown  in  Figure  9  reflect  a  major  portion  of 
the  systematic  uncertainty  affecting  the  performance  of  the  material, 
i.e.,  the  uncertainty  resulting  from  the  approximation  due  to  sampling. 
However,  there  are  additional  systematic  uncertainties  that  need  to  be 
accounted  for  in  a  complete  fragility  statement.  These  are  the  uncertain¬ 
ties  due  to  test  simulation  fidelity  and  test  calibration. 

When  the  systematic  uncertainties  can  be  assumed  to  be  lognormally 

distributed  with  constant  parameters,  these  components  of  systematic 

uncertainty  can  be  simply  root-sum-squared  together  to  provide  the  total 

uncertainty.  A  thorough  discussion  of  the  rationale  behind  this  assertion 

(2) 

is  given  in  the  report  of  the  previous  study 

The  difficulty  which  arises  in  the  implementation  of  this  strategem 
is  that  the  systematic  uncertainty  given  by  the  non-central  t-distribution 
is  not  constant  over  the  range  of  probabilities,  as  shown  in  Figure  10. 

The  factor  which  relates  the  confidence  bounds  varies  continuously  as  prob¬ 
ability  of  failure  varies. 

Figures  10  illustrates  the  variation  in  the  confidence  bounds 
given  by  the  non-central  t-distribution.  Figure  10  is  a  listing  of  the 
2.5,  50.0,  and  97.5  percent  confidence  curves  for  the  tests  of  MDAC 
graphite  epoxy  rings.  In  Figure  11,  the  data  have  been  transformed  to  the 
logar i thm/probabi 1 i ty  plane  so  that  a  lognormal  curve  appears  as  a 
straight  line.  The  straight  solid  line  in  the  center  of  the  plot  is  the 
best  estimate  (50-percent  confidence)  lognormal  distribution  based  on  the 
maximum  likelihood  estimators  printed  at  the  top  of  the  figure. 

The  curved  solid  lines  in  Figure  11  are  the  non-central  t  confidence 

bounds  for  2.5-  and  97.5-percent  confidence  (95  percent  of  the  population 

included).  Finally,  the  dashed  lines  in  the  figure  represent  constant  Kg 

factor  loci  used  to  approximate  the  2.5/97.5  confidence  bounds.  Note  that 

the  constant  K„  factor  which  determines  the  position  of  the  dashed  lines  is 
B 

based  on  matching  confidence  bounds  at  the  2.5-percent  confidence  level 
and  a  probability  of  failure  of  10  percent. 
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MEDIAN  =  2.4378 
K-FACTOR  =  1.5786 
SIZE  *  9 


MDAC  EPOXY  RINGS 


MEAN  *  0.8911 
STD  DEV  -  0.2329 
GROUPS  =  5 


TEST  LEVEL,  S  (UNCERTAINTY  FACTOR,  KB) 


PROB  CONFIDENCE 

(PF)  2.5  50.0  97.5 


0.10 

0.5915 

(  2.0062) 

1.00 

0.8229 

(  1.7229) 

2.00 

0.9244 

(  1.6343) 

3.00 

0.9946 

(  1.5815) 

5.00 

1.0980 

(  1.5135) 

10.00 

1.2749 

(  1.4186) 

15.00 

1.4064 

(  1.3616) 

20.00 

1.5173 

(  1.3207) 

25.00 

1.6165 

(  1.2889) 

30.00 

t  .7082 

<  1.2631) 

35.00 

1.7948 

(  1.2418) 

40.00 

1.8779 

(  1.2238) 

45.00 

1.9588 

(  1.2087) 

50.00 

2.0382 

(  1.1961) 

55.00 

2.1171 

(  1.1856) 

60.00 

2.1965 

(  1.1772) 

65.00 

2.2775 

(  1.1708) 

70.00 

2.3613 

(  1.1664) 

75.00 

2.4500 

(  1.1642) 

80.00 

2.5466 

(  1.1645) 

85.00 

2.6571 

(  1.1680) 

90.00 

2.7938 

(  1.1761) 

95.00 

2.9953 

<  1.1939) 

97.00 

3.1272 

(  1.2082) 

98.00 

3.2252 

<  1.2196) 

99.00 

3.3826 

(  1.2391) 

99.90 

3.8496 

(  1.3008) 

1.1868 

(  1.3008) 

1  .5437 

1.4178 

(  1.2391  ) 

1.756? 

1.5108 

<  1.2196) 

1.8426 

1.5729 

(  1.2082) 

1.9003 

1.6618 

(  1.193?) 

1.9841 

1.8086 

(  1.1761) 

2.1271 

1.9149 

(  1.1680) 

2.2366 

2.0039 

(  1.1645) 

2.3336 

2.0835 

(  1.1642) 

2.4256 

2.1577 

(  1.1664) 

2.5167 

2.2287 

(  1.1708) 

2.6093 

2.2983 

(  1.1772) 

2.7055 

2.3676 

(  1.1856; 

2.8070 

2.4378 

<  1.1961) 

2.9158 

2.5100 

(  1.2087) 

3.0339 

2.5857 

(  1.2238) 

3.1645 

2.6664 

(  1.2418) 

3.3111 

2.7542 

(  1.2631) 

3.4790 

2.8523 

(  1.2889) 

3.6763 

2.9656 

(  1.3207) 

3.9166 

3.1034 

(  1.3616) 

4.2256 

3.2859 

(  1.4186) 

4.6613 

3.5762 

(  1.5135) 

5.4125 

3.7783 

(  1.5815) 

5.9751 

3.9336 

(  1.6343) 

6.4288 

4.1915 

(  1.722?) 

7.2216 

5.0075 

(  2.0062) 

10.0461 

Figure  10.  Estimated  Distribution  tor  MDAC  Epoxy  Rings 
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Figure  11.  Transformed  Plot  of  Constant  K  Matched  at  10% 


The  system  designer  will  try  to  assure  that  the  system  will  exhibit 
an  adequate  probability  of  survival  at  a  medium  to  high  level  of  confidence. 
Because  Figure  11  is  plotted  in  terms  of  failure  probability,  it  is  neces¬ 
sary  to  work  on  this  plot  with  the  complements  of  the  desired  values  of 
probability  and  confidence.  In  other  words,  a  desired  probability-of- 
survival/conf idence  of  90/90  will  be  read  on  this  plot  as  probability-of- 
failure/conf idence  of  10/10. 

Confidence  bounds  of  2.5  and  97.5  percent  are  plotted  in  Figure  11 
because  these  levels  have  been  conventionally  accepted  as  the  levels  used 
to  define  a  constant  Kg  (see  equation  39) .  When  designing  for  confidence, 
it  is  likely  the  designer  would  work  to  less  stringent  confidence  levels 
if  possible. 

Matching  the  constant  Kg  lines  at  10-percent  PF  causes  an  over-conser¬ 
vative  bound  at  failure  probability  levels  of  50  percent  and  above.  On  the 
other  hand,  at  low  probabilities  of  failure,  the  constant  K_  bound  is  seen 
to  be  overly  optimistic.  The  log-probability  transform  of  Figure  11  is 
ideal  for  illustrating  the  difficulties  that  arise  in  trying  to  select  a 
constant  Kg.  However,  the  presentation  of  these  data  in  the  natural, 
untransformed  plane  as  shown  in  Figure  12  provides  a  better  perspective  of 
the  tradeoffs  involved.  The  discrepancies  are  here  not  so  exaggerated  as 
in  the  transform  plane. 

Figures  13  and  14  provide  a  similar  comparison  for  constant  Kg  lines 
matched  at  a  probability  of  failure  of  20  percent,  and  15  and  16  illustrate 
a  match  at  30  percent  probability  of  failure.  If  it  is  remembered  that  the 
system  designer  is  most  concerned  with  a  range  of  probabilities  of  failure 
from  about  50  percent  to  5  percent,  the  constant  Kg  line  matched  a  prob¬ 
ability  of  failure  of  20  percent  appears  to  provide  an  approximation  that 
would  be  adequate  for  many  design  problems. 

From  Figure  10,  the  95-percent  included  confidence  factor  at  a  prob¬ 
ability  of  failure  of  20  percent  is  seen  to  be  1.32.  This  is  the  approxi¬ 
mate  measure  of  systematic  uncertainties  due  to  sampling  effects.  Fitzgerald 
of  MDAC  estimated  that  differences  between  the  pulse  simulated  by  the 
magnetic  flyer  plate  in  the  test,  and  the  pulse  induced  by  exoatmospheric 

X-rays  would  cause  a  systematic  uncertainty  factor  of  1.20.  He  also 

( 2 ) 

estimated  calibration  uncertainties  to  yield  a  factor  of  1.10. 
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Figure  12.  Natural  Plot  of  Constant  Kn  Matched  at  10% 


— ■  eg 
CD  cn 
co  eg 

4  • 

o  o 


II  II 


cc  o 

UJ  I- 
E  CO 


C"  CD 

cn  O' 
•*  to 
•  • 

eg  — 


ii  ii 

cc. 

o 

Z  t- 

cc  o 
—  ac 
a  u. 
ui  I 


—  o 


O.S 


tfsoyj  aymiyj  *jj 


8 


Figure  14.  Natural  Plot  of  Constant  K„  Matched  at  20* P. 
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TEST  LEVEL 


These  elements  of  systematic  uncertainty  can  be  combined  to  give  a  total 
systematic  uncertainty 


Kb  =  exp  |  [<loge  1 . 32 }  2  +  (loge  1.20}2  +  (loge  1.10)2J2j 
=  1.41 

This  is  the  value  of  K_  for  aero-shell  spall  used  in  the  BASH  code 

o 

of  the  reentry  vehicle  presented  in  Volume  I. 


(40) 

analyses 
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4.0  CONDITIONAL  PROBABILITY  PAST 


The  FAST  code  has  been  applied  under  the  NHEP  program  to  assess  the 
hardness  of  reentry  vehicles  to  exoatmospheric  X-rays  and  to  determine  the 
associated  confidence  level.  Confidence,  C,  is  defined  as  the  estimated 
likelihood  that  at  least  a  prescribed  proportion,  P,  of  a  flight  of  reentry 
vehicles  will  have  a  given  fluence  capability,  0  p^.  In  running  the  FAST 
code  for  a  given  threat,  responses  are  simulated  at  several  fluence  levels 
by  Monte  Carlo  techniques.  Confidence  estimates  C^(0^,  P)  are  calculated 
for  each  fluence  at  a  prescribed  probability.  The  fluence  capability  0p/q- 
is  then  found  by  interpolation: 


0 


P/C 


=  0 


+  (0U  -  *1)  ( 


c  - 


w 


W  p>  "  C1{01' 


p>) 
t r 


(41) 


By  repitition  of  this  process,  fluence  capability  can  be  plotted  as  a 
function  of  threat  temperature,  T.  These  0-T  plots  are  the  desired  end 
produce  of  the  Nuclear  Hardness  Evaluation  Procedures. 

Early  in  the  NHEP  program,  the  question  was  raised  of  how  the  0-T 
curve  changes  given  one  or  more  successful  underground  test  of  a  reentry 
system,  subsystem,  or  component.  A  partial  answer  to  this  question  has 
already  been  obtained  during  a  previous  study.  In  that  study,  the  method 

of  conjugate  distributions  was  used  to  update  the  survivability-versus- 
confidence  curve  at  a  given  threat  and  fluence  level,  based  upon  r  success¬ 
ful  tests  out  of  n.  Assuming  that  P/C  curves  generated  by  the  FAST  code 
are  adequately  fitted  by  beta  distribution  functions  with  parameters  r' ,  n' 

then  the  post  test  distribution  was  estimated  by  a  beta  function  having 

(21) 

the  parameters  r"  =  r  +  r’;  n"  =  n  +  n’ . 

The  method  of  conjugate  distributions  has  a  number  of  limitations 
which  preclude  its  general  use  for  updating  0-T  plots.  These  limitations 
include : 

1)  Inability  to  correct  for  other  than  tested  fluence  levels; 

2)  Inability  to  correct  for  other  than  tested  threats; 

3)  Inability  to  account  for  less  than  perfect  simulation  of  the 

operational  environment  by  the  test  environment; 

4)  Inability  to  update  subsystem  or  systems  survivability 
based  on  component  testing;  and 
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5)  Questionable  applicability  of  the  beta  distribution  function 
for  all  systems  and  subsystems.  Poor  fits  have  occasionally 
been  observed  with  this  distribution. 

However,  these  limitions  have  been  overcome  by  a  new  version  of  FAST,  known 
as  Conditional  Probability  FAST,  FAST?. 

FAST 7  computes  conditional  probability,  defined  as  the  probability 
that  an  event  will  occur  given  the  condition  that  a  different  event  is  known 
to  have  occurred.  The  standard  notation  for  the  probability  that  event  A 
will  occur  given  that  event  B  has  occurred  is  P(A]B). 

Inputs  to  the  previous  versions  of  FAST  are  allowed  in  the  form  of 
series  systems,  such  as  SI  =  Cl  n  C2,  where  a  failure  of  either  component 
Cl  or  C2  fails  the  system;  or  parallel  systems,  such  as  S2  =  Cl  u  C2, 
where  a  failure  of  both  components  fails  the  system.  Parentheses  are  also 
allowed  in  combination  with  the  Boolean  operators  C\  and  u  to  build  fairly 
complex  logic. 

Parentheses  and  Boolean  operators  are  not  used  in  defining  conditional 
probabilities.  The  only  permitted  forms  for  conditional  probabilities  are: 


S3  =  SI | S2, 

(42) 

or 

S3  =  Sl|  SSI 

(43) 

or 

S3  =  Sl|  Cl 

(44) 

Systems  or  subsystems  defined  above  may  not  be  used  further.  However, 
these  restrictions  (discussed  below)  do  not  limit  the  general  applicability 
of  the  method,  since  any  series  or  parallel  event  can  be  represented  as  a 
system,  subsystem,  or  component  to  designate  a  complex  event. 

After  the  conditional  probability  has  been  properly  defined  in  the 
FAST 7  input,  the  code  computes  the  fluence  capability  by  the  following  steps 

1)  A  set  of  bias  variation  values  are  randomly  selected  in  the 
outer  iteration  loop; 

2)  The  code  computes  a  probability  of  survival  for  the  systems  or 
subsystems  representing  events  A  and  B;  and 

3)  The  code  locates  the  appropriate  cell  in  the  FAST  output 
histogram  for  event  A. 

For  example,  if  a  probability  of  survival  of  0.92  is  computed,  the  appro¬ 
priate  cell  might  be  represented  by  the  cell  boundaries  0.90  and  0.95, 
with  the  midpoint  0.925.  Instead  of  adding  a  one  to  the  sum  in  this  cell. 
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the  probability  of  survival  of  the  system  or  subsystem  representing  event  B 
is  added  to  the  sum.  Then,  a  new  set  of  systematic  or  bias  variations  are 
drawn  in  the  outer  iteration  loop,  and  steps  1  through  4  are  repeated. 

At  the  end  of  a  FAST7  run,  the  P/C  histogram  includes  the  estimated 
likelihoods,  C",  that  the  probability  of  survival  of  the  system  representing 
event  A  is  at  least  some  specified  value,  P,  given  that  a  system  represent¬ 
ing  event  B  has  survived.  Hence,  the  confidence  estimate,  C",  is  the  con¬ 
fidence  in  Pg  (A)  conditional  upon  survival  of  B.  In  fact,  event  B  need  not 
represent  a  successful  test.  A  test  failure  could  be  modeled  in  the  FAST 
code  by  using  the  reciprocal  of  the  fragility  parameter  and  the  reciprocal 
of  the  fragility  curve,  as  explained  below. 

Given  conditional  confidence  estimates,  C" ,  fluence  capability  is  com¬ 
puted  as  before 


0 

P/C" 


(  0U  -0L)  (C"  -  Cx  (02,  P>) 

Cu(0u,P)  -  C1  (0i;P) 


(45) 


Fluence-Threat  capability  plots  are  then  obtained  by  repeating  this  process 
for  other  threats. 

It  is  interesting  to  compare  the  method  conjugate  distributions  dis¬ 
cussed  above  with  the  method  of  FAST 7 .  The  method  of  conjugate  distribution 
requires  that  both  the  test  environment  and  the  environment  of  the  assess¬ 
ment  be  the  same.  The  FAST7  method  does  not  require  this  restriction  because 
the  system  representing  event  B  (the  test  outcome)  can  share  some  or  all  of 
the  uncertainties  of  event  A  (the  system  assessment)  without  having  the 
identical  environment.  Moreover,  simulation  uncertainties  can  be  modeled  in 
the  FAST7  assessment  by  including  an  extra  transfer  function  or  USER  sub¬ 
routine  with  the  component  inputs  which  describe  the  tested  failure  modes. 
Furthermore,  the  models  of  the  tested  failure  modes  need  not  include,  all  of 
the  failure  modes  of  the  system  A  assessment  model. 

The  FAST7  likelihood  estimates  do  not  rely  upon  beta  distribution  fits 
to  subsystem  and  system  survivability-confidence  statements,  and  they  can 
accurately  predict  the  entire  P/C  curve  provided  sufficient  Monte  Carlo 
iterations  are  performed. 
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5.0  CONCLUSIONS 


The  purpose  of  this  study  was  to  develop  and  demonstrate  new  techniques 
for  estimating  the  potential  benefits  of  testing  under  uncertainty.  A 
methodology  is  presented  for  estimating  the  reduction  in  the  systematic 
uncertainty  of  a  selected  component  resulting  from  testing,  and  for  eval¬ 
uating  the  system  benefit  associated  with  this  uncertainty  reduction. 


Two  different  approaches  for  the  analysis  of  the  effects  of  testing 
were  investigated  during  this  study.  The  first  is  a  method  for  analysis 
of  test  data  that  utilizes  maximum  likelihood  estimation  of  the  distribu¬ 
tional  parameters.  This  method  is  embodied  in  a  new  computer  code  named 
ELF,  Evaluation  of  Likelihood  Function.  The  second  approach  involves  the 
use  of  conditional  probability,  and  has  been  incorporated  into  a  new 
version  of  the  FAST  code  named  FAST 7 .  Both  of  these  computer  codes  can 
be  used  to  objectively  evaluate  systematic  uncertainties  based  on  test 
results. 

As  a  result  of  the  exploratory  work  performed  with  these  computer  codes 
during  the  present  study,  the  following  conclusions  can  be  drawn: 

•  The  ELF  code,  utilizing  maximum  likelihood 
estimation,  was  demonstrated  to  provide  a 
simple  and  effective  method  to  objectively 
measure  the  effects  of  testing  on  component 
uncertainty. 

•  The  FAST7  code,  utilizing  conditional  prob¬ 
ability,  was  shown  to  provide  a  general 
technique  for  tent  analysis  which  can  even 
account  for  uncertainties  in  test  simulation. 
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Air  Force  Systems  Command 
ATTN:  RSSE 
ATTN:  RSMA 
ATTN:  RSS 

Strategic  Air  Conmand 

Department  of  the  Air  Force 
ATTN:  DO XT 
ATTN:  XOBM 
ATTN:  XPFS 
ATTN :  XPQM 

DEPARTMENT  OF  ENERGY 

Department  of  Energy 

ATTN:  Document  Control  for  OMA/RD&T 

DEPARTMENT  OF  ENERGY  CONTRACTORS 


Effects  Technology,  Inc. 

ATTN:  R.  Parrise/M.  Rosen 
ATTN:  R.  Wengler/R.  Bick 

Ford  Aerospace  &  Communications  Corp. 
ATTN:  P.  Spangler 

General  Electric  Co. 

Re-Entry  &  Environmental  Systems  Div. 
ATTN:  P.  Cline 
ATTN:  D.  Edelman 
ATTN:  G.  Harrison 
ATTN:  N.  Dispenzierre 

General  Electric  Company-TEMPO 
ATTN:  DASIAC 

General  Research  Corp. 

ATTN:  R.  Rosenthal 


Lawrence  Livermore  Laboratory  Institute  for  Defense  Analyses 

ATTN:  Document  Control  for  L-96,  L.  Woodruff  ATTN:  Classified  Library 

ATTN:  Document  Control  for  L-125,  J.  Keller  ATTN:  J.  Bengston 

ATTN:  Document  Control  for  L-92,  C.  Taylor 

ATTN:  Document  Control  for  D.  Hanner  ION  Physics  Corp. 

ATTN:  R.  Evans 

Sandia  Laboratories 

Livermore  Laboratory  Kaman  AviDyne 

ATTN:  Document  Control  for  H.  Norris,  Jr.  ATTN:  R.  Ruetenik 

ATTN:  Document  Control  for  T.  Gold 
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DEPARTMENT  OF  DEFENSE  CONTRACTORS  (Continued) 

Kaman  Sciences  Corp. 

ATTN:  F.  Shelton 
ATTN:  T.  Meagher 
ATTN:  R.  Sachs/k.  O'Keefe 

Lockheed  Missiles  &  Space  Co.,  Inc. 

ATTN:  F.  Borgardt 

Lockheed  Missiles  &  Space  Co.,  Inc. 

ATTN:  0.  Burford 
ATTN:  R.  Walls 

Lockheed  Missiles  &  Space  Co.,  Inc. 

ATTN:  T.  Fortune 

Martin  Marietta  Corp. 


ATTN 

L.  Kinnaird 

McDonnell  Douglas  Corp. 

ATTN 

J .  Peck 

ATTN 

H.  Berkowitz 

ATTN 

L.  Cohen 

ATTN 

E.  Fitzgerald 

Pacific-Sierra  Research  Corp. 

ATTN:  G.  Lang 

Physics  Interiational  Co. 

ATTN:  , .  Shea 

Prototype  Development  Associates,  Inc. 
ATTN:  N.  Harington 
ATTN:  J.  McDonald 

R  &  D  Associates 

ATTN:  P.  Rausch 
ATTN:  E.  Field 
ATTN:  C.  MacDonald 
ATTN:  W.  Graham,  Jr. 


DEPARTMENT  OF  DEFENSE  CONTRACTORS  (Continued! 

Rand  Corp. 

ATTN:  R.  Rapp 

Science  Applications,  Inc. 

ATTN:  W.  Yengst 

Science  Appl ications,  Inc. 

ATTN:  W.  Layson 
ATTN:  W.  Seebaugh 

Southern  Research  Institute 
ATTN:  C.  Pears 

SRI  International 

ATTN:  G.  Abrahamson 
ATTN:  D.  Curran 
ATTN:  H.  Lindberg 

Systems,  Science  &  Software,  Inc. 

ATTN:  T.  McKinley 
ATTN:  R.  Duff 
ATTN:  G.  Gurtman 

Terra  Teck,  Inc. 

ATTN:  S.  Green 

TRW  Defense  &  Space  Sys.  Group 
ATTN:  L.  Donahue 
ATTN:  W.  Wood 
ATTN:  G.  Teraoka 
2  cy  ATTN:  P.  Dai/D.  Jortner 

TRW  Defense  &  Space  Sys.  Group 
ATTN:  V.  Blankenship 
ATTN:  W.  Polich 
ATTN:  J.  Farrell 
ATTN:  R.  Mortensen 
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